Suppression of undesired components in the measured spectra of spectrometers

ABSTRACT

The effects of unwanted components such as H 2 0 and CO 2  in spectral data measured by a spectrometer such as an FT-IR spectrometer are suppressed by a technique in which data representing high resolution spectra of the unwanted component is acquired, this data is modified so that its resolution matches that of the instrument, the results data is filtered to allow for perturbing effects of the sample and the resulting data is subtracted from the measured sample sectrum to provide corrected output data.

[0001] This invention relates to spectrometers and in particular relatesto suppressing undesired spectral components in the spectra obtainedfrom the spectrometers. The invention has particularly but not exclusiveapplication to FT-IR spectroscopy.

[0002] In for example an FT-IR spectrometer, infra-red or near infra-redradiation is directed from a source of such radiation towards a sampleunder investigation. Radiation transmitted by or reflected from thesample is received at a detector or receiver and the output of thedetector is processed by a signal processor in order to obtain thespectral characteristics of the sample. In carrying out measurements itis first necessary to obtain what is known as a background measurementthat is to say to the measure the background spectrum without a samplein place at the sample station. Subsequently measurements are made withthe sample in place and the desired sample spectrum is obtained from theratio of the measurement obtained with the sample in place to thebackground measurement.

[0003] A significant proportion of spectroscopy carried out in the midinfra-red range is concerned with spectra of solids and liquids whichhave absorbtion bandwidths in the tens of wave numbers. As aconsequence, many of the measurements are made at, for example, 4 cm−1or 2 cm−1 resolution. At such moderate resolutions water vapour havingline widths closer to 0.1 cm−1 is strongly under resolved and bands thatmight otherwise be saturated with absorbance well in excess of 1.0 arebroadened to the point where they show relatively low peak absorbance.Two major consequences result from this under resolution: The absorbancebecomes substantially non-linear with concentration causing theabsorbance spectrum shape to become a strong function of concentrationand in addition the lineshapes become entirely dominated by theinstrument lineshape function making the spectra have a differentcharacter according to the instrument type and set up. Moresignificantly since the instrument lineshape function may well beinfluenced by the sample or sampling accessory through vignettinganother beamed geometry disturbances the water vapour spectrum in thesample spectrum may not entirely resemble the nomunally similar spectrumin the background. That is to say the effects do not cancel out when thesample spectra and the background spectra are ratioed. The overallresult is that it is extremely difficult to subtract out the effects ofwater vapour consistently by any means of linear spectrum differencingtypically employed in this type of spectroscopy even if advancestatistical methods such as Principal Components Analysis are employed.A similar problem exists with other unwanted components such as those ofcarbon dioxide.

[0004] A further aspect of the problem is related to the overlap offeatures in the sample spectrum with features in the water vapourspectrum. Even when an exact spectrum of water vapour measured under thecurrent sampling conditions can be generated independently by somemeans, it is very difficult to estimate precisely the proportion ofwater vapour spectrum that must be subtracted out from the detectedspectra especially by automatic algorithmic approaches.

[0005] The present invention is concerned with a novel technique forsubtracting out from the measured sample spectrum undesired componentssuch as those arising from water vapour and carbon dioxide.

[0006] According to the present invention there is provided aspectrometer which comprises a source of analysing radiation, a detectorfor detecting radiation transmitted through or reflected from a sampleunder investigation, and a processing means for processing the output ofthe detector to produce spectral data relating to a sample underinvestigation, wherein the processor is arranged, in order to suppressthe effects of unrequired components such as water vapour or carbondioxide in the spectral data, to acquire reference data representinghigh resolution spectra of the unwanted component or components, tomodify said data so that its resolution simulates that of thespectrometer, to filter said data so as to allow for perturbing effectsin the sample spectrum and to subtract the resulting data from themeasured sample spectrum in order to provide a corrected output data.The acquired data may represent said high resolution spectra at aplurality of temperatures. The modification may comprise processing saidreference data so as to broaden the resolution to that which matches thesample spectral data being measured. The broadening may be carried outby a convolution technique in which the reference data are convolvedwith a computed line shape function appropriate to the spectrometer.These steps may be repeated for different reference data representingdifferent concentrations of unwanted component or components.

[0007] The processor may also be arranged to generate perturbed versionsof the broadened reference data to take into account variations in atleast some operating parameters such as temperature and optical linewidth.

[0008] The filtering may include creating a filter for filtering thespectra to emphasise higher resolution parts of the unrequiredcomponents. The filter may comprise a band pass filter. Filtering mayalso include carrying out a least square fit of the filtered samplespectral data to the filtered unwanted component spectra. Additionallythe processor may iterate the least square fit to remove those parts ofthe spectrum which have a poor fit.

[0009] The processor may be arranged to use the resulting co-efficientsto compute an unfiltered unwanted component spectrum. This is thensubtracted from the measured sample spectrum in order to obtain acorrected sample spectrum.

[0010] The invention will now be described now by way of. example andwith particular reference to the accompanying drawings. In the drawings:

[0011]FIG. 1 is a schematic illustration of an infrared spectrometer inwhich the present technique can be implemented and FIG. 2 is a flowchart illustrating the sequence of steps carried out by a processorarranged to operate in accordance with the present invention.

[0012] Referring to FIG. 1 of the drawings an infra-red spectrometer ofthe single beam type such as an FT-IR spectrometer in very generalterms, comprises a source of analysing radiation 10 which is arranged todirect analysing radiation to a sample station 12. Typically the sourceof analysing radiation will be associated with an interferometer of theMichelson type which produces a scanning beam by means of which thesample can be irradiated. Radiation from the sample station 12 isreceived at a detector or receiver 14 and the output of the receiver isprocessed by a processor 16 to provide data representative or thespectrum of a sample under investigation. Generally speaking aspectrometer of this type has an associated PC for carrying out at leastsome of the processing and enabling an operator to issue appropriateinstructions to operate the spectrometer. The way in which aspectrometer of the type shown in FIG. 1 operates in order to obtain aspectrum will be known to those skilled in the art and will thereforenot be described in any more detail since it is unnecessary forunderstanding the present invention.

[0013] Embodiments in accordance with the present invention areconcerned with routines which are operated by the processor in order tosuppress or cancel out the effects of undesired components such as watervapour or carbon dioxide in the measured sample spectra. It is envisagedthat such routines can be operated by a wide range of spectrometersprovided that the processing means of the spectrometer has sufficientcapability to run the software involved in carrying out the routines.The description given here is directed in the main to the way in whichthe invention can be implemented in FT-IR spectroscopy.

[0014] Initially a general description of the method will be givenfollowed by an implementation in more specific terms. The descriptionwill refer to removing the effects of water vapour but it should beunderstood that the technique applies equally well to supparesion ofother undesired components such as carbon dioxide.

[0015] In very general terms the processor is arranged to remove orsuppress the effects of any water vapour components in the spectrum bythe following steps.

[0016] 1. Generating a properly broadened approximately representativeset of spectra for the unwanted component which span the range ofnon-linear behaviour expected in the instrument.

[0017] 2. Generating perturbed spectra to take into account a degree ofvariability in lineshape parameters; and

[0018] 3. Iterative fitting using a combination of filtering techniques.

[0019] The initial step in the method is to acquire stored highresolution spectral data relating to water vapour. This should havesufficient resolution to accurately characterize the natural lineshape.It is only when the line shape is properly resolved that the behaviourof the bands can be properly represented as peak absorbances increaseand the non-linearity of the transmission values becomes significant.Such data is available commercially and one example which can be used isa set of peak tables known as (HITRAN). Significant parameters of suchspectral data are atmospheric pressure which under normal circumstancesis responsible for line broadening and in the present case will beunderstood to be one atmosphere, and a representative gas temperaturewhich in the present case is taken to be 23° C., a typical laboratoryenvironment.

[0020] Starting from this single spectrum the method generates spectrafor three different concentrations of water at a pathlength typical ofthe actual pathlength encountered in practice eg 25 cm. Theconcentrations selected are 25% relative humidity, (RH) 50% RH and 100%RH thereby spanning the range of concentrations typically encountered inpractice. The simple way of obtaining these spectra is to square thetransmittance spectrum of a base 25% RH spectrum to obtain the 50% RHspectrum and then repeat the process in order to obtain the 100% RHspectrum. An alternative way of doing this would be by a conventionallogarithmic scaling technique.

[0021] These three principal spectra are then broadened intransmittances in order to simulate approximately the broadening thatoccurs in the spectrometer itself. In the present case, ie a FourierTransform spectrometer, the spectrum is broadened on a linear wavenumber abscissa using an FT lineshape function arising from acombination of the interferogram scan length used in the instrument andan apodisation function employed. The broadening can be performed eitherin spectrum space with a convolution or interferogram space bymultiplying an envelope function. In the present example the operationis carried out in the spectrum space. In the next step the spectrum isbroadened on a logarithmic wave number abscissa using a rectangular lineshape function associated with a perfectly behaved circular Jacquinotstop. The Jacquinot stop controls the divergence of rays passing throughthe interferometer of the spectrometer and effectively controls itsresolving power. The associated line width is proportional to wavenumber and hence on a logarithmic scale appears to have constant width.There are a number of ways in which this calculation can be performed.In the present case it is achieved by computing the mean value over theline width from the area under spectrum, using a variable line width. Itshould be understood that the Jacquinot stop can also introduce aspectrum shift. In the present case this is already substantiallycalibrated out by use of a centre broadening function.

[0022] At this stage the process has produced three principal broadenedspectra at the various concentrations. The next step is to generatethree additional minor components which allow for variation from thecomputed spectra. The first is a similarly broadened spectrum at 50% RHbut at 10° C. higher temperature to allow for temperature variation. Thesecond is at 50% RH at the standard temperature broadened with aslightly smaller Jacquinot stop function to allow for imperfect optics.The third is a spectrum shifted in wave number by a small constant toallow for calibration errors. In each case, the minor component actuallyemployed in the subsequent processing is the difference between theperturbed spectrum and the principal 50% RH component.

[0023] In the majority of cases the six components that it is to say thethree major components and the three minor components should besufficient to implement a technique in accordance with the presentinvention. However, if necessary it would be possible to iterate the fitusing more appropriate starting parameters once estimates for theparameters have been made from the fit components, for exampletemperature and path length. In some cases it may be possible to operatewith less than six components especially if an iterative approach ofrefining the initial parameter estimates is adopted. An iterativeapproach works best for single beam spectra where the water vapourspectrum is directly represented as opposed to a ratio spectrum whereonly small differences in water vapour may be present. Once havingrefined the parameters from the single beam subsequent elimination ofthe effects of water vapour is best carried out on the ratio spectrumsince the ration itself eliminates a significant part of the watervapour effect.

[0024] Having established the components which can be used to accuratelyreconstruct the expected water vapour transmittance spectrum, it isnecessary to determine the proper coefficients for the fit in thepresence of the unknown and potentially significant determining factorsin the data arising from, for example, the spectrum of the sample. Thesemay be many times stronger than the spectrum of the water vapour. It isfound in practice that the problem of subtracting out the water vapourremains linear in absorbence even though the construction of the watervapour spectrum itself is a non-linear problem. Thus, in the presentexample the spectra are converted to absorbence before the followingoperations.

[0025] The first step in reducing the interference from the samplespectra is to filter all the data, that is the calculated componentspectra and the sample spectrum, with a band pass filter that acceptsdata centred around 10 cm−1 resolution while rejecting data at higherand particularly lower resolution. The parameters of this filter arebased on the general properties of infra-red spectra of solids andliquids typically measured in the applications of concern. Theunderlying principle is that of a matched filter.

[0026] In order to create such a filter typical sample spectra obtainedfrom a commercial library are Fourier transformed and the square rootsum of squares of the real and imaginary components calculated in orderto give an idea of the distribution of the information present acrossthe range of resolution. In the present case this indicates the amountof data at resolutions better than 10 cm−1 is relatively small. Asimilar calculation for water vapour shows that being a gas there isconsiderable information up to the resolution limit of the spectrum. Theratio of the data for the water vapour versus the library data shows abroad peak centred around 10 cm−1. The shape of this peak approximatesto a gaussian and this function forms the basis of the matched filter,and is analytic in both transform spaces.

[0027] It should be noted that the matching of the filter is notespecially critical and good results can. be obtained using a simplefirst derivative filter. However, a gaussian function approximatesclosely to the ideal for a general case.

[0028] The above described resolution filtering reduces the effects ofinterference to water vapour least square fits significantly. However,it is still not sufficient to produce a satisfactory result in manysituations. In order to achieve this an additional step of iterativeweighting of the fit in the spectrum domain is carried out.

[0029] In this the initial weight function employed is the measuredtransmittance spectrum itself. The reason for this is that regions oflow transmittance have poor signal to noise ratio and a good fit cannotbe expected in these regions. The weighted least squares fit of thefiltered data is computed and the residual spectrum calculated. Theweighted root mean square residual value is calculated and then theweight function is modified to set zero weight for those portions of thespectrum where the residue exceeds four times the rms value. Thisprocess effectively blanks out regions of the spectrum where the fit isparticularly bad. The fit is iterated and the process repeated until thefit is sufficiently improved. There may be a compromise between thenumber of iterations and the severity of the blanking and in practice itis found that the final result is better with less severe blanking andmore iterations.

[0030] Having calculated the best parameters of the fit using filtereddata the process reconstructs the full water vapour spectrum using thesame fit parameters with the unfiltered data. The measured samplespectrum is then divided in transmittance by the water vapour spectra inorder to elimininate the water vapour component.

[0031] It is believed that the above description will enable areasonably competent computer programmer to implement a program forcarrying out the process described.

[0032] In order to assist in this respect the following description inconjunction with the flow chart of FIG. 2 is included as a specificexample for eliminating an unwanted component in accordance with thepresent invention.

[0033] This program effectively has two inputs these being as follows:

[0034] 1 The first is the target spectrum or the measured spectrum ofthe sample from which the water vapour component is to be subtracted.The spectrum is typically a low resolution spectrum measured at around 4cm−1 resolution in transmittance, digitised at a constant wave numberinterval at around 2× over-sampling, that is at 1 cm−1 interval.

[0035] 2 This is the high resolution reference spectra of water vapourat two difference temperatures. These spectra typically resolve thenatural line width of the water vapour under measurement conditions ofthe target spectrum, ie around 0.1 cm−1 for normal pressure, and arestored in transmittance at intervals of {fraction (1/64)} cm−1. The twotemperatures should lie on either side of the temperature of the targetspectrum and the water vapour concentration should be set around 25% RHat the path length of the target spectrum.

[0036] Referring now to FIG. 2 the first step illustrated at 20 is thereading of the target spectrum that is to say the spectrum measured bythe spectrometer. The next step illustrated at 21 is to read in thelower temperature reference spectrum, ie the second input referred toabove.

[0037] The next step shown at block 22 is to compute the lineshapecorresponding to the Fourier transform by which the target spectrum wasgenerated. This line shape is calculated by computing eitheranalytically or numerically the Fourier transform of the apodisationfunction applied to the interferogram from which the target spectrum wasgenerated. In the case of no apodisation the lineshape will be a sincfunction of an appropriate width.

[0038] The next step shown at block 23 is to broaden the referencespectrum to the same nominal resolution as the target spectrum byconvolving with the computed lineshape. Convolving involves multiplyingthe first part of the reference spectrum by a short line function andsumming the products. Then the alignment of the two spectra is shiftedalong by one point and the process repeated. The sums form a newspectrum which is the convolution of the original spectrum with thelineshape function.

[0039] The next step is to reduce the sampling of the broaden referencespectrum to the same interval as the target spectrum by dropping datapoints. It should be noted that once the lineshape has been broadeneddata points can be discarded without loss of information.

[0040] The next step, as illustrated at 25, is to square the originallow temperature reference spectrum to produce a 50% RH spectrum and thisis broadened in a manner similar to that described for the initialreference data.

[0041] Referring to block 26 this is then repeated to give the 100% RHbroadened reference spectrum. At this stage there exist three spectrafor that cover the concentration range of the target spectrum. The 50%RH spectrum will be referred to as the principal component.

[0042] The next step as illustrated at 27 is to read in the highertemperature reference spectrum. This is squared and broadened in amanner similar to that already described to a give a broadened 50% RHhigher temperature fourth component. This will be used later to allowfor any temperature variation in the target spectrum. Each of the fourcomponents are then further broadened by a rectangular shape functionwhich represents the ideal effect of the spectrometers optical aperture.This rectangular function has a theoretical fractional optical frequencywidth dnu/nu=1−cos (theta) where that is the semi-angle subtended by theJacquinot stop aperture at the collimating mirror.

[0043] Theta=arctan (j-stop diameter/2* focal length)

[0044] The convolution is computed directly by numerical integration ofthe area under this rectangle when multiplied into the operand spectrum.

[0045] At this stage the process has computed four of the six requiredcomponent spectra. The next step is to generate in a similar manner aj-stop broadened version of the principal component but with 10%narrower rectangular functions. This allows for some experimentalvariability of optical line width.

[0046] The principal component is then interpolated to generate a sixthcomponent with a small wave number shift proportional to opticalfrequency. This allows for some tolerance in the frequency calibrationof the target.

[0047] Then all six components are converted to absorbance asillustrated at block 30 and the principal component subtracted from thelast three so that the process handles only minor perturbations producedby temperature, j-stop and shift.

[0048] The next step, shown in 31 is to produce a guassian-shaped bandpass digital filter function of width (l/e) of 0.25 the Nyquistfrequency and centred at 0.25 the Nyquist frequency. This filter can begenerated analytically and serves to emphasize high resolutioninformation unique to the water vapour spectrum. In the next step 32this filter is convolved with all components and also with the targetspectrum. Then a gaussian filter is computed which has a similar widthwhich is centred at zero frequency (33). This filter is convolved withthe unfiltered principal component to generate a weight spectrum (34)The filter so generated effectively smooths the absorbance peaks of thewater vapour spectrum to generate a function that resembles the envelopeof the peaks. The weight function is further multiplied by theunfiltered transmittance target spectrum.

[0049] The next step illustrated at 35 is to carry out a least squaresfit of the six filtered components to the filtered target data using theweights. This composite weight function ensures closeness of fitwherever the water vapour is likely to be most intrusive.

[0050] The next step shown by block 36 is to compute the square of theresidual spectrum multiplied by the weight spectrum. The residualspectrum is the difference between the target and the fitted spectrum.The fitted spectrum is computed by summing the component spectra scaledby the coefficients derived in the least squares fitting process.

[0051] Referring to block 37 the next step is to compute the rootweighted mean square residue. This number is the analog of the rmsresidue in an unweighted fit.

[0052] Then the process computes a weighted residual spectrum asillustrated at block 38 and then identifies data points in the weightedresidual spectrum which are greater than 4× the root weighted meansquare residue and zeros the corresponding weights in the weightingspectrum. This process identifies data points in the target that arebeyond the acceptable range and discards them as illustrated at 39.

[0053] The process is then iterated (40) from the least square fit steptypically six times in order to refine the fit and to make sure that themajority of the unacceptable data has been removed.

[0054] The process then uses the fit coefficients generated in the finaliteration to compute an equivalent of the fitted spectrum from theunweighted unfiltered components. (See block 41.)

[0055] This reconstructed water vapour spectrum is then subtracted fromthe unfiltered target spectrum (42). The reconstructed spectrum shouldaccurately match the water vapour component in the target spectrum whichshould then subtract out cleanly. The result is the sample spectrumwithout the water vapour component.

[0056] By way of general comments the effects of water vapour are mostprominent in a well separated spectrum regions for example the regionscentred around 1600 cm−1 and the region around 3700 cm−1. In theorysince these are all part of the same water vapour spectrum, one set offit parameters should suffice for all regions. However modelling ofthese spectrum may not be exact and it may be necessary to treat theregions independently.

[0057] At higher optical frequencies the instrument line shape functioncan become increasingly dominated by geometric and optical factors inthe beam path. The method described above relies for its basis spectraon a reasonably close approximation to ideal behaviour since theJacquinot stop is assumed to give rise to a rectangular line shape.Although the fit is not particularly sensitive to exact details of thisline shape, being dominated for the most part by FT line shape, it maybe preferable in some situations to use an alternative broadeningfunction for the Jacquinot stop effect.

[0058] The parameters of the least squares fit can yield diagnosticinformation on instrument performance. For example the overall strengthof the water bands in the background spectrum can be used to indicatethe state of purge of the instrument, the amount of shifted spectrumemployed in the fit can be used as a check on the abscissa calibration.It could in principle be used as the primary abscissa calibration,although this may not be wholly desirable.

[0059] The method described above relates to suppression of water vapourcomponents. The principles described can be applied equally well toremoval of carbon dioxide components. However, account needs to be takenof differences arising from the difference in spectrum and resolutiondistribution. Carbon dioxide has a more regular band strength structurethat is easily obscured by underresolving but on the other hand it hasless overlap with typical sample spectra, particularly in the strong2350 cm−1 band. This tends to place more emphasis on the spectrumweighting and less on resolution filtering.

What is claimed is:
 1. A spectrometer which comprises a source ofanalysing radiation, a detector for detecting radiation transmittedthrough or reflected from a sample under investigation, and a processingmeans for processing the output of the detector to produce spectral datarelating to a sample under investigation, wherein the processor isarranged, in order to suppress the effects of unrequired components suchas water vapour or carbon dioxide in the spectral data, to acquirereference data representing high resolution spectra of the unwantedcomponent or components, to modify said data so that its resolutionsimulates that of the spectrometer, to filter said data so as to allowfor perturbing effects in the sample spectrum and to subtract theresulting data from the measured sample spectrum in order to provide acorrected output data.
 2. A spectrometer according to claim 1 whereinthe acquired data represent said high resolution spectra at a pluralityof temperatures.
 3. A spectrometer according to claim 1 wherein themodification comprises processing said reference data so as to broadenthe resolution to that which matches the sample spectral data beingmeasured.
 4. A spectrometer according to claim 3 wherein the broadeningis carried out by a convolution technique in which the reference dataare convolved with a computed line shape function appropriate to thespectrometer.
 5. A spectrometer according to claim 4 wherein these saidsteps are repeated for different reference data representing differentconcentrations of unwanted component or components.
 6. A spectrometeraccording to claim 1 wherein the processor is arranged to generateperturbed versions of the broadened reference data to take into accountvariations in at least some operating parameters such as temperature andoptical line width.
 7. A spectrometer according to claim 1 wherein thefiltering includes creating a filter for filtering the spectra toemphasise higher resolution parts of the unrequired components.
 8. Aspectrometer according to claim 7 wherein the filter comprises a bandpass filter.
 9. A spectrometer according to claim 8 wherein thefiltering also includes carrying out a least square fit of the filteredsample spectral data to the filtered unwanted component spectra.
 10. Aspectrometer according to claim 9 wherein the processor is also arrangedto iterate the least square fit to remove those parts of the spectrumwhich have a poor fit.
 11. A spectrometer according to claim 9 whereinthe processor is arranged to use the resulting co-efficients to computean unfiltered unwanted component spectrum.
 12. A method of operating aspectrometer which comprises a source of analysing radiation, a detectorfor detecting radiation transmitted through or reflected from a sampleunder investigation, and a processing means for processing the output ofthe detector to produce spectral data relating to a sample underinvestigation, said method being carried out in order to suppress theeffects of unrequired components such as water vapour or carbon dioxidein the spectral data, and comprising the steps of acquiring referencedata representing high resolution spectra of the unwanted component orcomponents, modifying said data so that its resolution simulates that ofthe spectrometer, filtering said data so as to allow for perturbingeffects in the sample spectrum and subtracting the resulting data fromthe measured sample spectrum in order to provide a corrected outputdata.
 13. A method according to claim 12 wherein the acquired datarepresent said high resolution spectra at a plurality of temperatures.14. A method according to claim 12 wherein the modification stepcomprising processing said reference data so as to broaden theresolution to that which matches the sample spectral data beingmeasured.
 15. A method according to claim 14 wherein the broadening iscarried out by a convolution technique in which the reference data areconvolved with a computed line shape function appropriate to thespectrometer.
 16. A method according to claim 15 including repeatingsaid steps for different reference data representing differentconcentrations of unwanted component or components.
 17. A methodaccording to claim 12 including generating perturbed versions of thebroadened reference data to take into account variations in at leastsome operating parameters such as temperature and optical line width.18. A method according to claim 12 wherein the filtering step includescreating a filter for filtering the spectra to emphasise higherresolution parts of the unrequired components.
 19. A method according toclaim 18 wherein the filter comprises a band pass filter.
 20. A methodaccording to claim 18 wherein the filtering step also includes carryingout a least square fit of the filtered sample spectral data to thefiltered unwanted component spectra.
 21. A method according to claim 20including iterating the least square fit to remove those parts of thespectrum which have a poor fit.
 22. A method according to claim 20including using the resulting co-efficients to compute an unfilteredunwanted component spectrum which is then subtracted from the measuredsample spectrum.